Adjusting an image for video conference display

ABSTRACT

A video conferencing system contains one or more display adjusting components, whereby an object to be displayed can be adjusted to appropriately fit various sized display screens. A display adjusting component is contained within the sending client, which adjusts the image of the object to be appropriately displayed to one or more receiving clients. The receiving clients also contain a display adjusting component, which can further adjust the image of the object to be displayed, as necessary. The multimedia conferencing server of the video conferencing system also contains a display adjusting component, which negotiates parameters of the sending and receiving clients. Any of the display adjusting components can function alone, or in any combination together. A method, and computer-readable media which contain computer-readable instructions to perform a method, of adjusting an image for video conference display are also described.

PRIORITY

This application is a continuation of U.S. patent application Ser. No.13/889,044 having issued as U.S. Pat. No. 9,342,752 on May 17, 2016,which is a continuation of U.S. patent application Ser. No. 12/722,323having issued as U.S. Pat. No. 8,471,889 on Jun. 25, 2013, the entiretyof each being incorporated by reference herein.

INTRODUCTION

A conventional video conference system displays the same proportionalsize of an image, regardless of the size of the display screen.Therefore, a large display screen would project a large life-sizedimage. This frequently results in the image being out of focus. Inaddition, a large sized image of a person can have the appearance ofbeing overbearing or making conference viewers uncomfortable. Likewise,a conventional video conference system displays a small life-sized imageonto a small display screen, which may be difficult to view clearly.

SUMMARY

Embodiments of the invention are defined by the claims below. Ahigh-level overview of various embodiments of the invention is providedto introduce a summary of the systems, methods, and media that arefurther described in the detailed description section. This summary isneither intended to identify key features or essential features of theclaimed subject matter, nor is it intended to be used as an aid inisolation to determine the scope of the claimed subject matter.

Embodiments of the invention use one or more display adjustingcomponents to adjust the image to appropriately fit various sizeddisplay screens. A large display screen will down-size the displayedimage and include more background video content. Likewise, a smalldisplay screen will up-size the displayed image and include little or nobackground video content, thereby providing clearer detail and moreattention on the image. Adjusting an image to an appropriate fit isachieved by utilizing one or more display adjusting components at asending client, a receiving client, or a multimedia conferencing server.The display adjusting component of the sending client will usuallyadjust the image first to its capabilities, then the display adjustingcomponents of the receiving clients will further adjust the image to fittheir respective display screens. However, this particular order ofsteps is not a requirement of the invention. The multimedia conferencingserver also comprises a display adjusting component, which negotiatesfrom known factors of the sending and receiving clients, and works toovercome any limitations of either the sending or receiving clients.Embodiments of the invention contemplate using any of the displayadjusting components alone or in any combination with each other.

Displayed objects comprise, for example, an image of a participant of avideo conference meeting, an inanimate object, or an electronic displayof text, graphics, or other matter to be discussed as part of the videoconference meeting.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

Illustrative embodiments of the invention are described in detail below,with reference to the attached drawing figures, which are incorporatedby reference herein, and wherein:

FIG. 1 is a block diagram of a multimedia conferencing system accordingto embodiments of the invention;

FIG. 2 is an illustration of an image outline system according toembodiments of the invention; and

FIG. 3 is a flow diagram for a method of adjusting an image for videoconference display according to embodiments of the invention.

DETAILED DESCRIPTION

Embodiments of the invention provide systems, methods, andcomputer-readable media with computer-readable instructions embodiedthereon for adjusting an image of a video conference display. The terms“step,” “block,” etc., might be used herein to connote different acts ofmethods employed, but the terms should not be interpreted as implyingany particular order, unless the order of individual steps, blocks,etc., is explicitly described. Likewise, the term “module,” etc., mightbe used herein to connote different components of systems employed, butthe terms should not be interpreted as implying any particular order,unless the order of individual modules, etc., is explicitly described.

Throughout the description of different embodiments of the invention,several acronyms and shorthand notations are used to aid theunderstanding of certain concepts pertaining to the associated systems,methods, and computer-readable media. These acronyms and shorthandnotations are intended to help provide an easy methodology forcommunicating the ideas expressed herein and are not meant to limit thescope of any embodiment of the invention.

Embodiments of the invention include, among other things, a method,system, or set of instructions embodied on one or more computer-readablemedia. Computer-readable media include both volatile and nonvolatilemedia, removable and non-removable media, and media readable by adatabase and various other network devices. Computer-readable mediacomprise computer storage media. By way of example, and not limitation,computer-readable media comprise media implemented in any method ortechnology for storing information. Examples of stored informationinclude computer-useable instructions, data structures, program modules,and other data representations. Media examples include, but are notlimited to, information-delivery media, Random Access Memory (RAM), ReadOnly Memory (ROM), Electronically Erasable Programmable Read Only Memory(EEPROM), flash memory or other memory technology, Compact Disc (CD)ROM, digital versatile discs (DVD), holographic media or other opticaldisc storage, magnetic cassettes, magnetic tape, magnetic disk storage,and other magnetic storage devices. These technologies can store datamomentarily, temporarily, or permanently. The computer-readable mediainclude cooperating or interconnected computer-readable media, whichexist exclusively on a processing system or distributed among multipleinterconnected processing systems that may be local to, or remote from,the processing system. Combinations of any of the above should also beincluded within the scope of computer-readable media.

An operating environment for devices and interfaces used for embodimentsof the invention include a processing system with one or more high speedcentral processing unit(s) (CPU), or other types of processors, and amemory system. The embodiments of the invention are described below withreference to acts and symbolic representations of operations orinstructions that are performed by the processing system, unlessindicated otherwise. Such acts and operations or instructions arereferred to as being “computer executed,” “CPU executed,” or “processorexecuted.” The memory locations where data bits are maintained arephysical locations that have particular electrical, magnetic, optical,or organic properties corresponding to the data bits.

Embodiments may be generally directed to multimedia conferencing systemsarranged to provide meeting and collaboration services to multipleparticipants over a network. Some multimedia conferencing systems may bedesigned to operate with various packet-based networks over aninterconnected computing network, such as the Internet or World Wide Web(WWW), to provide web-based conferencing services. Such implementationsare sometimes referred to as web conferencing systems. Other multimediaconferencing systems may be designed to operate for a privateinterconnected computing network, business, organization, or enterprise,and may utilize a multimedia conferencing server.

A multimedia conferencing system may include, without limitation, amultimedia conferencing server or other processing device arranged toprovide web conferencing services. For example, a multimediaconferencing server may include a server meeting component operative tocontrol and mix different types of media content for a meeting andcollaboration event, such as a web conference. A meeting andcollaboration event may refer to any multimedia conference eventoffering various types of multimedia information in a real-time or liveonline environment.

In an embodiment, the multimedia conferencing system may further includeone or more computing devices implemented as meeting consoles. Eachmeeting console may be arranged to participate in a multimedia event byconnecting to the multimedia conference server. Different types of mediainformation from the various meeting consoles may be received by themultimedia conference server during the multimedia event, which in turndistributes the media information to some or all of the other meetingconsoles participating in the multimedia event. As such, any givenmeeting console may have a display with multiple media content views ofdifferent types of media content.

FIG. 1 is a block diagram representing a multimedia conferencing system100 that can be used in conjunction with the invention. FIG. 1 is justone example of a multimedia conferencing system, and other multimediaconferencing systems can be used without departing from the scope of theinvention. The multimedia conferencing system 100 may comprise, or formpart of, a wired communications system, a wireless communicationssystem, or a combination of both. For example, the multimediaconferencing system 100 may include one or more elements arranged tocommunicate information over one or more types of wired or wirelesscommunications links.

In an embodiment, the multimedia conferencing system 100 may be arrangedto communicate, manage, or process different types of information, suchas media information and control information. Examples of mediainformation may generally include any data representing content meantfor a user, such as voice information, video information, audioinformation, image information, textual information, numericalinformation, application information, alphanumeric symbols, or graphics.Media information can also be referred to as “media content.” Controlinformation may refer to any data representing commands, instructions,or control words meant for an automated system. For example, controlinformation may be used to route media information through a system, toestablish a connection between devices, or instruct a device to processthe media information in a predetermined manner.

In an embodiment, multimedia conferencing system 100 includes amultimedia conferencing server 110. The multimedia conferencing server110 may comprise any logical or physical entity that is arranged toestablish, manage, or control a multimedia conference call between alocal meeting console 120 and any number of remote meeting consoles 125over a network 130. The local meeting console 120 and each of the remotemeeting consoles 125 comprise one or more clients. The multimediaconferencing server 110 may comprise or be implemented as any processingor computing device, such as a computer, a server, a server array orserver farm, a work station, a minicomputer, or a mainframe computer.The multimedia conferencing server 110 may comprise or implement acomputing architecture suitable for communicating and processingmultimedia information.

The multimedia conferencing system 100 of FIG. 1 may include aconference room 140 to hold meetings, which include events having localparticipants 150-1 and 150-2 through 150-n, where n equals an integer,located internally to the conference room 140. The multimediaconferencing system 100 also includes remote participants locatedexternally to the conference room 140, which communicate with theconference room 140 via the one or more remote meeting consoles 125. Theconference room 140 may include various multimedia input devicesarranged to capture media content from the conference room 140 andstream the media content to the multimedia conferencing server 110. Thevarious multimedia input devices may include one or more video cameras160 and an array of microphones 165. The video cameras 160 capture videocontent, including video content of the local participants 150 presentin the conference room 140. The video content is stream fed to themultimedia conferencing server 110 via the local meeting console 120.Similarly, the array of microphones 165 capture audio content, includingaudio content from the local participants 150 present in the conferenceroom 140, which is stream fed to the multimedia conferencing server 110via the local meeting console 120. The local meeting console 120 mayinclude various media output devices, such as a display or videoprojector to show one or more graphical user interface views with videocontent and one or more speakers to disseminate audio content receivedfrom the remote participants via the multimedia conferencing server 110.A similar arrangement of the video camera 160, the array of microphones165, and the video display and speakers, described for the conferenceroom 140, may also be present at each of the remote locations, accordingto the size and configuration of the various remote locations.

The multimedia conferencing system 100 of FIG. 1 also illustrates adisplay adjusting component 170 as part of the local meeting console120. In an embodiment of the invention, an image of each of theparticipants 150 can be captured by one or more of the video cameras160. The display adjusting component 170 calculates the optimal size ofthe facial outline of a participant 150, based upon the size of each ofthe video conference displays of the meeting consoles 120 and 125.Calculations include determining the size and geometry of the image, anddetermining the distance of the image from the image capturing camera160 by range finding techniques. A more detailed description forobtaining the facial outline is described below, with reference to FIG.2. As an example, the optimal size of the facial outline is adjusted bythe display adjusting component 170, such that a proportionately smallerimage will be displayed on a larger video conference display. Theproportionately smaller facial outline eliminates displaying anoversized facial image, which may be somewhat out of focus, or appear tobe overbearing or of uncomfortable size. The proportionately smallerfacial image could include more background as part of the video display.As another example, the optimal size of the facial outline is adjustedby the display adjusting component 170, such that a proportionatelylarger image will be displayed on a smaller video conference display.Since the viewing area of a small display screen is limited, the totalvideo display area will primarily comprise the facial image with lessbackground video content. An embodiment of the invention also comprisesdetermining which participant 150 in the conference room 140 ispresently speaking. Therefore, the image of the participant 150 who ispresently speaking, which is also referred to as “the talking head,” canbe displayed in larger proportion in comparison to images of the otherdisplayed participants 150, or “the talking head” could be displayedalone.

The displayed image can also be adjusted or resized by a displayadjusting component 175 of the receiving remote meeting console 125. Asan example, an image of the sending client from the local meetingconsole 120 is adjusted by the display adjusting component 175 of theone or more remote meeting consoles 125. The principles described abovewith regard to display adjusting component 170 of the local meetingconsole 120 are also applicable to the display adjusting component 175of each of the remote meeting consoles 125. Therefore, an image can beadjusted or resized by either the display adjusting component 170 or thedisplay adjusting component 175, or a combination of both displayadjusting components. Using a combination of both display adjustingcomponents would be advantageous when video camera capabilities arelimited at one or more locations and an optimum image display cannot beachieved solely by the display adjusting component of one meetingconsole.

In an embodiment of the invention, the multimedia conferencing server110 comprises a display adjusting component 180, as illustrated inFIG. 1. The display adjusting component 180 adjusts and resizes imagesfrom both the local meeting console 120 and any remote meeting consoles125. The display adjusting component 180 negotiates from known factorsof both the sending client and the receiving clients. The displayadjusting component 180 communicates with the local meeting console 120and the remote meeting consoles 125 via an interconnected computingnetwork 130, where the interconnected computing network 130 could be aprivate LAN or WAN network or the Internet, for example. The displayadjusting component 180 has at least the same capabilities as displayadjusting component 170 or display adjusting component 175, as discussedabove.

The display adjusting component of the sending client will usuallyadjust first to its capabilities, then the display adjusting componentof the receiving client will further adjust as necessary. For example,the sending client may adjust the image to a larger size for displayonto a small display screen. However, the clarity or definition may needto be improved, which can be accommodated by the receiving client. In analternative embodiment, the display adjusting component of the servercould do all of the adjusting. Combinations of any two or all displayadjusting components could be utilized with the invention in order tooptimize the displayed image.

FIG. 2 illustrates a system by which an image outline can be achieved.An image 200 depicting the face of a person is shown. A plurality offeature points 210 are identified and positioned to indicate a readilyrecognizable image. In the illustrated example of FIG. 2, the featurepoints 210 are positioned to indicate the border of the eyebrows, eyes,nose, and mouth. These feature points 210 are grouped into connectedsets 220, such as a facial outline. These connected sets 220 can becompared to a model object shape using, for example, the Active ShapeModel technique. A model of the shape of an object is constructed byanalysis of a set of example shapes. A shape model can be used toconstrain a given shape to the range of plausible relative featurepositions. These shapes are typically generated by manually annotatingpreferred feature positions on a set of exemplar images of an object.Principle components analysis is used to compactly model the range ofallowable shapes in terms of a mean shape. Other systems of recognizingand identifying an object can be used without departing from the scopeof the invention.

Adjusting and displaying an image of an object can be extended tonon-person objects also, including inanimate objects. Video conferencedisplays of adjusted images, according to embodiments of the inventiondescribed herein, can range from microscopic images, such as medical orscientific micrographs, to very large geographic images, such aslandscapes or skyscraper buildings. Adjusted images of detailed objects,such as building plans, are also included. An embodiment of theinvention also contemplates zooming in or out of the image, or panningaround a total image to a specific point of interest or discussion. Alive or on-site video stream feed captured by means of a camcorder, forexample, is also contemplated by embodiments of the invention. The liveor on-site video stream feed is adjusted as necessary, according toembodiments of the invention, then displayed to all participants of thevideo conference.

A method 300 of adjusting an image for video conference display will nowbe described, with reference to FIG. 3. A video conference meeting isusually held within a conference room, but other environments areincluded within the scope of the invention. The dimensions of eachconference room are inputted in step 310 for subsequent calculations.The dimensions of the local conference room, such as conference room140, as well as the conference rooms at any remote locations, areinputted. This information is saved in a computing system, such as themultimedia conferencing server 110, or it could be saved with eachassociated meeting console, such as local meeting console 120 and remotemeeting consoles 125. The dimensions of each display screen of eachassociated conference room are inputted in step 320. Multiple conferencerooms and associated multiple display screens are contemplated herein,as well as multiple sizes of display screens. An embodiment of theinvention includes the display screen of a wireless device, such as acellular phone or a personal digital assistant. This information is alsosaved in a computing system, such as the multimedia conferencing server110, or it is saved with each associated meeting console, such as localmeeting console 120 and remote meeting consoles 125.

An object that is to be displayed to the one or more display screens isdetermined in step 330. The object to be displayed could be the image ofa person, and in particular, could be an image of the person who ispresently speaking. This could be determined by voice activation sensorsworking in conjunction with one or more video cameras within theconference room. The object to be displayed could also be a non-personimage or an inanimate object image. Inanimate objects of multiple sizes,dimensions, locations, and categories are within the scope of theinvention described herein, as well as multiple types of electronicdisplays, such as textual information, numerical information,application information, alphanumeric symbols, and graphics. An exampleof adjusting an electronic display image, such as an outline of ameeting agenda, could include viewing more of the electronic display ina single viewing on a large display screen, and viewing less of theelectronic display in a single viewing on a small display screen. Thiswould provide an optimal reading size for each of the different sizeddisplay screens. In addition, the object or objects to be displayedcould comprise an on-site video stream feed.

The image of the object to be displayed is adjusted in step 350, suchthat the displayed image fits appropriately to each of the displayscreens, which may vary in size. A conventional video conference systemdisplays the same proportional size of the image, regardless of the sizeof the display screen. As an example, the image of a person may occupy50% of the total display screen area. Therefore, a large display screenwould project a large life-sized image. This frequently results in theimage being out of focus. In addition, a large sized image of a personcan have the appearance of being overbearing or making conferenceviewers uncomfortable. Likewise, a conventional video conference systemdisplays a small life-sized image onto a small display screen, which maybe difficult to view clearly.

Embodiments of the invention utilize one or more display adjustingcomponents in order to adjust the image to appropriately fit each of thevarious sized display screens. Therefore, the displayed image will bedown-sized for a large display screen and include more background videocontent. Likewise, the displayed image will be up-sized for a smalldisplay screen with little or no background video content, therebyproviding clearer detail and more attention on the image. Adjusting adisplayed image to an appropriate fit is achieved by one or more displayadjusting components. The local meeting console comprises a displayadjusting component and each remote meeting console usually comprises adisplay adjusting component. The display adjusting component of thesending client will usually adjust the image first to its capabilities,then the display adjusting component of the receiving client willfurther adjust the image, but this is not a requirement of theinvention. The multimedia conferencing server also comprises a displayadjusting component, which negotiates from known factors of the sendingand receiving clients, and works to overcome any limitations of eitherthe sending or receiving clients. Embodiments of the inventioncontemplate using any of the display adjusting components alone or inany combination with each other.

One or more computer-readable media containing computer-readableinstructions embodied thereon that, when executed by a computing device,perform a method of adjusting an image for video conference display isprovided. This method of adjusting an image for video conference displaymay contain some or all of the steps discussed above.

Many different arrangements of the various components depicted, as wellas components not shown, are possible without departing from the spiritand scope of the several embodiments of the invention. Embodiments ofthe invention have been described with the intent to be illustrativerather than restrictive.

It will be understood that certain features and subcombinations are ofutility and may be employed without reference to other features andsubcombinations and are contemplated within the scope of the claims. Notall steps listed in the various figures need be carried out in thespecific order described.

The invention claimed is:
 1. A projection system for video conferences,the system comprising: a centralized server that facilitates a videoconference between two or more local user devices, wherein thecentralized server includes a display adjusting component that:generates outlines of images, the images comprising a video stream ofthe video conference, identifies a focus image of the video conference,compares an outline of the focus image to models for various types ofobjects, adjusts the focus image based on the comparison, andcommunicates the adjusted focus image with instructions for display ateach of the two or more local user devices during the video conference.2. The projection system of claim 1, wherein the models includeportraits of a face or outlines of articles.
 3. The projection system ofclaim 1, wherein the display adjusting component utilizes display sizeof each of the two or more local user devices to adjust the focus image.4. The projection system of claim 2, wherein the articles include one ormore medical images, geographic images, buildings, or design plans. 5.The projection system of claim 1, wherein the display adjustingcomponent modifies a size of an outline corresponding to one or morefeature points of the focus image based on distance between an imagecapture device and an element represented by the focus image.
 6. Theprojection system of claim 1, wherein the display adjusting componentmodifies a size of an outline of one or more feature points of the focusimage when the one or more feature points are associated with voice dataof the video conference.
 7. The projection system of claim 6, whereinthe display adjusting component decreases the size of the outline of theone or more features points identified in the focus image when the oneor more features points are not associated with voice data of the videoconference.
 8. The projection system of claim 1, wherein the displayadjusting component identifies a size and a geometry of each of one ormore feature points identified in the focus image.
 9. The projectionsystem of claim 8, wherein the display adjusting component locates eyes,eye brows, and a mouth based on the size and the geometry of each of theone or more feature points identified in the focus image.
 10. One ormore non-transitory computer-storage media configured to execute amethod of projecting an image for video conference display, the methodcomprising: receiving, at a centralized server, a video stream from alocal user device; generating outlines of images, wherein the imagescomprise the video stream of a video conference facilitated by thecentralized server; for each one of the images: comparing an outline ofthe image to one or more shape models of objects; adjusting the imagebased on the comparison, wherein the outline of the image is constrainedusing at least one of the one or more shape models of objects; andcommunicating the constrained outline of the image and instructions todisplay the adjusted image at the local user device.
 11. The media ofclaim 10, wherein the one or more shape models include portraits of aface.
 12. The media of claim 10, wherein the one or more shape modelsinclude a range of allowable shapes that are constrained by a meanshape.
 13. The media of claim 10, wherein the one or more shape modelsinclude outlines of an article.
 14. The media of claim 13, wherein thearticle includes medical images, geographic images, buildings, or designplans.
 15. The media of claim 13, further comprising identifying a sizeand a geometry of feature points in the image.
 16. The media of claim15, further comprising locating eyes, eye brows, and a mouth based onthe size and the geometry of the feature points in the image.
 17. Themedia of claim 16, further comprising increasing the size of outlinescorresponding to the feature points of the image when the elementrepresented by the image is associated with speech data.
 18. The mediaof claim 10, further comprising modifying a size of the outlinecorresponding to feature points of the image based on the distance anelement represented by the image is from an image capture device. 19.The media of claim 10, further comprising modifying a size of theoutline corresponding to feature points of the image based on whether anelement represented by the image is associated with voice data.
 20. Acomputer-implemented method for projecting an image for video conferencedisplay, the method comprising: at a server facilitating a videoconference between a plurality of local user devices: receiving a videostream of the video conference between the plurality of local userdevices; generating outlines of images of the video stream as the videostream is received, the images comprising the video stream of the videoconference; identifying a focus image of the video conference; for eachone of the images of the video stream: comparing an outline of the imageto models for various types of objects, and communicating the outlinewith instructions for display until the object represented by theoutline is no longer the focus of the video conference.